Many companies and organizations operate computer networks that interconnect numerous computing systems to support their operations, such as with the computing systems being co-located (e.g., as part of a local network) or instead located in multiple distinct geographical locations (e.g., connected via one or more private or public intermediate networks). For example, data centers housing significant numbers of interconnected computing systems have become commonplace, such as private data centers that are operated by and on behalf of a single organization, and public data centers that are operated by entities as businesses to provide computing resources to customers. Some public data center operators provide network access, power, and secure installation facilities for hardware owned by various customers, while other public data center operators provide “full service” facilities that also include hardware resources made available for use by their customers. However, as the scale and scope of typical data centers have increased, the tasks of provisioning, administering, and managing the physical computing resources have become increasingly complicated.
The advent of virtualization technologies for commodity hardware has provided benefits with respect to managing large-scale computing resources for many customers with diverse needs, allowing various computing resources to be efficiently and securely shared by multiple customers. For example, virtualization technologies may allow a single physical computing machine to be shared among multiple users by providing each user with one or more virtual machines hosted by the single physical computing machine, with each such virtual machine being a software simulation acting as a distinct logical computing system that provides users with the illusion that they are the sole operators and administrators of a given hardware computing resource, while also providing application isolation and security among the various virtual machines. Furthermore, some virtualization technologies are capable of providing virtual resources that span two or more physical resources, such as a single virtual machine with multiple virtual processors that spans multiple distinct physical computing systems. In many cases, the functionality and usability of applications that run on such platforms may rely extensively on network communications with other parts of the provider network, and/or with external entities such as clients or third parties.
In network-based computing environments, multiple computers may be employed to manage incoming service requests. Load balancers may be used to distribute the service requests across the multiple computers. Distributing the service requests across the computers helps optimize resource utilization, maximize throughput, and avoid overload of any one of the computers. Typically, a load balancer receives a request from a client device, selects one of the multiple computers to service the request, and routes the request to the selected computer, which processes the request and issues a response. Load balancers generally attempt to distribute the request load evenly across all of the service computers in an effort to prevent any individual computer from being overloaded with requests. However, many customers of the network-based computing environment provider may require fault tolerant systems, such that no transactions are lost and full system state is preserved in the event of failure. Conventional load balancers do not provide fault tolerance.